Method for visualizing a bone

ABSTRACT

A method and a corresponding system are provided. The method comprises steps of providing 2D images and subsequently detecting outlines of a primary structure in each of the images. A visual representation of the 2D images is generated and the 2D images are then arranged as 2D slices in a 3D visual representation. To this end, at least two of the 2D images are taken at different imaging angles. The method provides a 3D visual representation of a region of interest comprising a primary structure to support a spatial sense of a user.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a national phase entry under 35 U.S.C. § 371of International Application No. PCT/US2016/047487 filed Aug. 18, 2016,published in English, the disclosure of which is incorporated herein byreference.

BACKGROUND OF THE INVENTION

The invention relates to the field of computer based assistance ofsurgery. In particular, the invention relates to a method of automaticimage processing including 3D visualization for supporting a spatialsense of a user. The method may be implemented as a computer programexecutable on a processing unit of a suitable device.

In every surgery where intraoperative imaging is used it is a challengeto accurately perform the necessary steps of a procedure of treating abone fracture. Usually, almost each step requires an iterative processbased on several X-ray images. A significant radiation exposure is thusproduced. The amount of radiation may be highly depended on the know-howand skill of the physician.

Some imaging systems may provide 3D visual representations of a surgeryarea or a region of interest, such as a region comprising a bonefracture. Such 3D visual representations may contain valuableinformation about the current spatial position of an implant in relationto a bone and/or one or several bone fragments. Based on the informationa surgeon may determine the next surgery steps or may determine the needof further (re)positioning of the implant. 3D visual representations ofa region of interest often rely on several 2D images, which on the onehand often requires enhanced radiation exposure of a patient and on theother hand causes enhanced computational expense.

Accordingly, there may be need for an improved assistance during asurgical procedure.

BRIEF SUMMARY OF THE INVENTION

It may be seen as an aspect of the invention to provide a method anddevice for more efficiently assisting in performing a surgicalprocedure. It would be highly beneficial to reduce the amount ofradiation to which a patient—as well as a surgeon—is exposed during asurgical procedure, to reduce computation time needed for providingvisual representations of a surgical area or region of interest, and/orto more efficiently visually represent relevant information on thesurgical site to the surgeon.

The mentioned aspects are solved by the subject-matter of each of theindependent claims. Further embodiments are described in the respectivedependent claims.

According to a first aspect of the invention, a method for visualizing abone, for instance a femur or a hip bone, is provided. The methodcomprises the step of providing 2D images of the bone, wherein at leasttwo of the 2D images are taken at different imaging angles. In a furthermethod step, outlines of a primary structure are detected in the 2Dimages. Based on the outlines, a visual representation of the 2D imagesis subsequently generated. Finally, a visual representation of the 2Dimages is arranged in form of 2D slices in a 3D visual representation.Thereby, the arrangement of the 2D slices in the 3D visualrepresentation is based on the different imaging angles under which the2D images are taken. A purpose of the 3D visual representation can beseen in supporting a spatial sense of a user. By “spatial sense of auser” it is meant that a person viewing the image can determine thelocation and orientation of a bone, bone fragment or implant from spaced2D slices forming a partial 3D visual representation of the imagedregion based on the user's prior knowledge.

The 2D images may be X-ray images, ultrasound-images, images taken withmagnetic resonance imaging or taken with any other imaging method. Themethod relies on provision of at least two 2D images taken fromdifferent angles. Accordingly, also a plurality of more than two 2Dimages taken from a plurality of more than two angles may be providedand processed within the method. The arrangement of the 2D images as 2Dslices in a 3D visual representation may, for instance, be conceived as2D pages of an opened book. More generally, the arrangement of the 2Dimages as 2D slices in a 3D representation may be conceived asintersecting 2D planes in a 3D image space. The arrangement of the 2Dplanes in the 3D space corresponds to the angles under which thecorresponding 2D images are taken, respectively.

2D images can be taken from different angles, for instance when an imageis taken with a C-arm imaging device and the C-arm is then rotated andanother image is taken. It is preferred within the invention that theangles under which different 2D images are taken are larger than 10°,respectively. More preferably, the angles under which different 2Dimages—which are subsequently provided within the method—differ by atleast 15°, or differ by more than 30°. This helps to make sure that the2D slices given in the 3D visual representation are inclined by thoseangles relative to each other, such that feature/parts of objectscomprised in the 2D slices sufficiently differ from each other. If theangles would be too close to each other, the features/parts of objectsdepicted in corresponding 2D slices would be similar and not of muchsupport for a spatial sense of a user, who wants to reconstruct with hisspatial sense a full 3D view of a region of interest.

The 3D visual representation according to the invention differs fromconventional 3D representations in that the 3D visual representation isbuilt from a collection of at least two 2D planes—or more generally 2Dsurfaces—and the space in between the respective 2D surfaces is leftempty. Thus as few as two 2D planes can provide sufficient input for auser to determine essential information about the imaged region. Each 2Dsurface contains image information of the considered region of interestaccording to a specific angle or perspective and a specific 2D layer orlevel of the region of interest. In the 3D visual representation, the atleast two 2D surfaces are arranged according to the angles orperspectives and the position of the respective layer of the region ofinterest. For example, a 3D visual representation according to theinvention can be imagined as a collection of intersecting and/orparallel arranged 2D planes in 3D position space. Corresponding imageinformation is comprised on the 2D planes, respectively. The remainingspace between the 2D planes is left empty, i.e. no image information isprovided or given for this space.

In other words, the invention relates to a method for automatedgeneration of a grid model of a bone. The invention does not provide afull 3D image reconstruction of a bone or a region of interest, butprovides essential information on the bone or region of interestcomprised in 2D slices, which map parts of the full 3D region ofinterest. In this way, computational costs for full 3D imagereconstruction can be reduced. At the same time, the spatial sense of auser can be stimulated to reconstruct the full or essential parts of the3D information, such that the necessary or important information isavailable due to a combination of the provided reduced 3D visualrepresentation and the spatial sense and experience of a user.

According to an embodiment of the invention, a primary structure in the2D images may be given by at least one of a bone, and a bone fragment.

According to an embodiment of the invention, a primary structure in the2D images may be given by at least one of an implant, and a referencebody.

A reference body may comprise a plurality of fiducial members ormarkers, most preferably at least three such markers that are visible toan imaging system, which is used to provide the 2D images. It mayfurther be preferred that the fiducial markers comprise spheres that arevisible to the imaging system. The reference body may be used indetermining a spatial dimension and position of an implant. For example,an implant may include the reference body or may be positioned in apredefined location in relation to the reference body. Both thereference body and the implant are detected or recorded in an image, forinstance an X-ray or fluoro image. By means of the correctidentification and registration of the reference body—in particular thepreferably comprised fiducial markers —, the actual spatial dimensionand position of the implant can then be determined.

The reference body may be directly attached to an anatomical structure,e.g. may be in contact with an outer surface of a body part of interest.The reference body may also be indirectly coupled to an anatomicalstructure, for example via a handling tool for inserting an implant. Onthe other hand, the reference body may be at least a part of an implant.In other words, an implant which is adapted to be fixed at or in a bonemay comprise elements which can be identified in an image of the bone orat least a section of the bone so that geometrical aspects may bedetermined based on the identified elements. For example, the elementsmay define points so that two elements may define a line or an axis, orthe elements may define a contour so that a centre axis may bedetermined.

Further, the reference body may be integrated into an aiming device for,e.g. supporting an insertion of a locking screw through a bore in aleading end of a bone nail, wherein the aiming device may be adapted tobe coupled to a handling tool for inserting the bone nail. Therefore,the aiming device may be adapted to be coupled to a trailing end of thebone nail and may extend outside the body of a patient as far as thebone nail extends inside the bone so that at least a portion of theaiming device can be visible in an image of the section of the boneincluding the leading end of the bone nail. Such an aiming device isdescribed and shown in U.S. Pat. No. 8,685,034, the disclosure of whichis incorporated herein by reference.

As used herein, the term “anatomical structure” refers to anything at abone and in particular to a geometrical aspect of a bone, i.e. a point,a line, an arc, a centre point, an axis, a cylinder surface, a ballsurface, or the like. For example, a geometrical aspect of a femur maybe the outer surface of the femur head, an axis defined by the neckbetween shaft and femur head, a longitudinal axis of the femur shaft, amost distal point on the bone surface, a line defined by the centrepoints of the condyles, or a line defined by the most posterior pointsat the condyles. It will be understood that the other bones provideother and/or comparable suitable geometrical aspects.

According to a further embodiment of the invention, the method furthercomprises the step of detecting at least three markers in the 2D images.Thereby, a marker may be a reference body or a part of a reference body,an implant or a bone shape. For instance, a marker may be a radiopaquesphere, which is part of a reference body attached to an implant. In afurther method step a spatial arrangement of the primary structure basedon the position of the at least three markers can be determined. Thearrangement of the 2D images as 2D slices in the 3D visualrepresentation is then based on the determined spatial arrangement.

According to an embodiment of the invention, the method furthercomprises the step of classifying the primary structure into a class ofimplants, a class of bones, a class of bone fragments and/or a class ofreference bodies. The method-step of arranging the visual representationof the 2D images in form of 2D slices in the 3D visual representationmay then be based on the classification of the primary structure.

For instance, primary structures corresponding to different classes—forexample corresponding to a class of implants and a class of bones—may berepresented in different colors or different shading and/or hatching inthe 3D visual representation. The 3D visual representation may compriseall detected primary structures, or the 3D visual representation maycomprise only a subset of the primary structures, which are contained ina specific class or in specific classes. In the latter case, thespecific class or the specific classes to be visualized in the 3D visualrepresentation may be selectable by a user.

According to an embodiment of the invention, the visualization of theprimary structure is limited to at least one of the classes of implants,bones, bone fragments and/or reference bodies.

Reducing the number and class of primary structures to be visualized inthe 3D visual representation according to the embodiment may support auser to focus on the essential information contained in the acquired 2Dimages and gathered in the 3D visual representation.

According to a further embodiment of the invention, the 3D visualrepresentation is rotatable, such that the 2D images, which arerepresented as 2D slides in the 3D visual representation, can be viewedfrom different viewpoints. This may further support a spatial sense of auser.

According to an embodiment of the invention, the visualization of theprimary structure may be based on a detection of a predetermined surgerystep. Thereby, detection of a predetermined surgery step can be based ona number and a position of primary structures comprised in the 2Dimages.

Detection of a predetermined surgery step may be based on imageprocessing of the 2D images comprising detection of variousobjects/devices in the 2D images. The detected objects/devices in the 2Dimages may be indicative of the step in the workflow during surgery.With the detection of the objects/devices in the 2D images it may bedetermined which step in the workflow is currently being performed by auser and which step(s) has(have) been performed before that.

More specifically, detection of a predetermined surgery step maycomprise steps for identifying a current state of elements detected inthe 2D image. Here, “current state” means first of all a position andorientation of the detected element. For instance, the position andorientation of the reference body can be determined due to the specificdistribution of fiducial markers forming of being attached to thereference body. With respect to an instrument, like a gauge or a drillor a K-wire, the position may be detected in relation to the referencebody and/or to an anatomical structure. A “current state” of aninstrument may also include a deformation or bending of the instrument.Furthermore, a “current state” may indicate the appearance of theinstrument and/or of an implant or sub-implant in the respective 2Dprojection image.

Based on the identified state of the detected elements, a state ofprogress of the procedure of e.g. fracture treatment may be determined.For example, information provided by a database with results of thepreviously performed may be compared to the position and number ofelements in the 2D image, with the database including data defining eachstep out of a sequence of steps necessary to perform e.g. a fracturetreatment procedure. For example, the steps may be defined in thedatabase by the respective state of the elements which elements areinvolved in the particular step, so that information extracted from the2-D projection image can be compared with information received from thedatabase.

The step following the identified step out of the sequence of steps inthe database may be used to provide or to derive information which stepshould be performed next.

According to an embodiment of the invention, the 2D images are X-rayimages.

As stated above, the invention is not limited to X-ray images and imagesobtained with other imaging techniques than X-ray imaging may be usedwithout departing from the scope of the invention.

A second aspect of the invention relates to a system for visualizing abone. The system comprises a detection unit and a processing unit. Thedetection unit is configured to provide 2D images of the bone. Thereby,at least two 2D images should be taken from different imaging angles.The detection unit is further configures to detection outlines of aprimary structure in the 2D images. Based on the outlines, theprocessing unit is configured to generate visual representations of the2D images. The processing unit is further configured to arrange thevisual representations of the 2D images as 2D slices in a 3D visualrepresentation. The arrangement of the 2D slices in the 3D visualrepresentation is based on the different imaging angels under which the2D images are taken. It is intended that the 3D visual representationsupports a spatial sense of a user.

A detection unit may be an imaging unit for providing images of ananatomical structure. For instance, a detection unit can be an X-rayimaging unit of a C-arm apparatus. A processing unit may be a processingdevice for processing data, such as a processing unit comprised in acomputer. Such computer may be integrated together with its processingunit in a system for visualizing a bone according to the embodiment. Theprocessing unit as part of a computer may also be an external deviceconnectable to a detection unit, which is configured to provide imagesof an anatomical structure.

The detection unit of the imaging system may be comprised in an imagingapparatus such as a C-arm apparatus. Other examples of imaging systemsproviding a detection unit for providing 2D images may be one of anX-ray apparatus, an ultrasound imaging device or a magnetic resonanceimaging device.

A third aspect of the invention relates to a computer program. Thecomputer program is configured to perform any one or any combination ofthe above described method steps.

A fourth aspect of the invention relates to a computer readable medium,on which an above described computer program is stored. It may beconsidered as a gist of the invention to provide a 3D visualrepresentation of a bone, which relies on a set of several 2D imagesonly and supports a spatial sense of a surgeon without relying on a full3D reconstruction of a region of interest. In other words, the 3Dpartial representation of the bone, bone fragment or implant providessufficient information to allow the surgeon to use his or herimagination to fill in the missing data utilizing on the surgeon'sknowledge of the region of interest which is based on the surgeon'sexperience and expertise. The surgeon knows from experience (or earlierimages) the 3D form and shape of, for example, the femur of a patient.To reconstruct the position and orientation of the femur from an X-rayimage he then does not need an image of the femur comprising the fullshape and form of this bone, but only requires certain characteristicpoints (like an outline or the position of the head and theintertrochanteric line of the femur) may be enough for him. From hisexperience, and using his spatial imagination, he can reconstruct the 3Dorientation of the femur by seeing the location of the characteristicpoints only. Hence, as a surgeon typically knows the actual shape/formof the objects depicted in the X-ray images, it is not necessary thatthe images contain the full information on the shape/form of theobjects. An outline or characteristic points of a particular object canbe enough to allow a skilled surgeon to complete, by his spatialimagination, the full object.

By using the method or the associated system according to the invention,a radiation dose to which a patient and a surgeon may be exposed duringa surgery due to the number of images to be taken for obtaining a 3Dvisual representation of a region of interest may be significantlyreduced. The method and the system may require fewer images because only2D slices in a 3D visual representation are visualized and a full 3Dreconstruction of the region of interest is not needed. Accordingly,method and system are therefore safer for patient and surgeon. Inaddition, computation power and computation time may be significantlyreduced as no full 3D reconstruction is required.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flow chart of steps of a method according to anembodiment of the invention.

FIG. 2 shows an example of a monitor visualization of a 2D X-ray imageaccording to an embodiment of the invention.

FIGS. 3A and 3B show schematic illustrations of a monitor visualizationof a 3D visual representation according to an embodiment of theinvention.

FIG. 4 shows another flow chart of steps of a method according to anembodiment of the invention.

FIG. 5 shows a schematic illustration of a system according to anembodiment of the invention.

FIGS. 6A and 6B are schematic visualizations regarding a projection of areference body.

DETAILED DESCRIPTION

The flow chart in FIG. 1 illustrates method steps performed inaccordance with embodiments of the invention. It will be understood thatthe steps described may be major steps, wherein these major steps mightbe differentiated or divided into several sub-steps. Furthermore, theremight be also sup-steps between the steps. Consequently, groups of stepsmay be related to a major aspect which may also be performed separately,i.e. independent from other steps or groups of steps.

It is noted that some steps are described as being performed “ifnecessary”. This is intended to indicate that those steps may beomitted. It is in particular noted that a computer program elementaccording to an embodiment of the invention may comprise sets ofinstructions to automatically recognize if a step is necessary or not,and to automatically proceed with the next actually necessary step.

With reference to FIG. 1, the exemplary method starts with providing 2Dimages of a region of interest in step S1. A region of interest may be abone and the anatomic structures surrounding the bone. The images may beX-ray images. Alternatively, the images may be ultrasound images,magnetic resonance images or may be given by any other type of imagesacquired in medical diagnostics and/or therapy. In method step S2outlines of a primary structure are detected in each of the 2D images ofthe bone. Thereby, an outline may be the shape of a primary structure inthe respective image. Accordingly, an outline can be given by a contourline of the respective primary structure in the image. Detection of anoutline can be carried out by e.g. comparing the grey-level value ofneighbouring pixels or clusters of pixels in an image, therebydetermining contour lines and in particular determining contour lines,which define the edge or boundary of a structure in the image, i.e. theboundary of a structure with respect to another structure depicted inthe image.

A primary structure can be a bone and/or a bone fragment. Further, aprimary structure can be given by an implant and/or a reference body. Incontext of the invention, a “primary structure” is a structure ofinterest during a surgical procedure. For instance, a femur or a hipbone, a corresponding implant and/or a bone screw can be primarystructures in a 2D image provided in step S1 of a method according tothe invention. Accordingly, a non-primary structure is a structure,which is of minor of no relevance in a specific surgical procedure.Tissue surrounding e.g. a hip bone, may, for instance be a non-primarystructure, which can be of minor relevance for a surgeon during fixingof an implant in a hip bone.

In optional method step S′, the detected primary structures can, ifnecessary, be classified into different classes, e.g. a class ofimplants, a class of bones and/or a class of markers or referencebodies. It may be possible that only those primary structures comprisedin a specific selectable class or several selectable classes arevisually represented in later method steps S3 and S4.

In method step S3, visual representations of each of the 2D imagesprovided in step S1 are generated, based on the outlines detected instep S2. These visual representations of 2D images are subsequentlyarranged in a 3D visual representation in method S4. This is done byarranging the 2D images as corresponding 2D slices in a 3Dvisualisation. Each slice is arranged according to the angles underwhich the 2D image is taken. An example of such an arrangement isdescribed in more detail in context of FIGS. 3A and 3B below.

In FIG. 2, an example of a visualisation of a 2D image on a monitor ordisplay is shown. The image comprises a bone 1 with a corresponding bonefragment or bone part 2. Further, an implant 3 is shown, as well as adrilling tool 4. A plurality of markers 5 is visible in FIG. 2. Thelatter markers allow determination of a spatial order and/or arrangementof the further structures visible in the image. With respect to FIGS. 6Aand 6B a method is explained below that allows determination of aspatial arrangement from one single image.

In FIG. 3A a schematic example of a 3D visual representation 10displayed on a monitor or display 6 is shown. The 3D visualrepresentation 10 comprises several 2D planes or 2D slices 20. On each2D plane corresponding image information, i.e. information from acorresponding 2D image is displayed. Thereby, the 2D planes are arrangedin the 3D visual representation reflecting the angles under which the 2Dimages were acquired. Accordingly, there is an angle α between thesurface normal of the two planes in FIG. 3A.

The 3D visual representation may be rotatable. The possibility ofrotating the representation, i.e. the representation of the planes inFIG. 3A is indicated by arrows 21 and 22. This possibility may furtherprovide assistance to a surgeon, supporting his spatial sense to capturethe 3D arrangement of a surgical area without the need to display all 3Dinformation of the area but only slices or cuts through this area.Exemplarily, in order to put the idea across, a cuboid 30 is shown as a3D structure in FIG. 3A. Only the solid lines 31, 32, which show theintersection of the cuboid 30 with the 2D planes, are displayedaccording to the invention. The dashed lines of the cuboid 30 are notshown, but only given in FIG. 3A to illustrate the idea.

FIG. 3B shows a further schematic example of a 3D visual representation10 displayed on a monitor 6. The 3D visual representation comprisesseveral 2D planes, which are arranged in parallel to each other with acertain space between them. Arrows 21 and 22 indicate that the 3D visualrepresentation may be rotatable. Further, the solid lines 31, 32 arecontour lines of a cuboid 30, similar to the cuboid shown in FIG. 3A.However, in contrast to FIG. 3A, in FIG. 3B only the contours or theoutline of the cuboid in the 2D planes 20 is shown.

Thus in FIG. 3A, the surgeon will know that the object visualised on thedisplay (reference sign 6) is a cuboid (reference sign 30). However, thecuboid will not be displayed (the dashed lines will not be visible tothe surgeon according to the invention). What will actually be shown onthe display are the thick lines 31 and 32 only. Using his spatial sense,the surgeon can however reconstruct the position and orientation of thecuboid 30, although only the thick black lines 31 and 32 are shown onthe display. In FIG. 3B a similar situation as in FIG. 3A is shown. Herethe hashed planes are visualised on the display. The surgeon knows thatthese planes belong to a cuboid and can deduce, using his imagination(spatial sense), the position and orientation of the cuboid, relying onthe depicted information.

The flow chart in FIG. 4 illustrates method steps performed inaccordance with another embodiment of the invention. In method step S1,2D images are provided and in each of these images outlines of at leastone primary structure are detected in subsequent method step S2.Optionally, the detected primary structures can then be classified intoone of the classes of implants, bones, bone fragments, reference bodiesand/or markers.

In the subsequent step S2′, at least three markers are detected in eachof the 2D images. Based on the position and angles under which thesemarkers are detected in each of the 2D images, a spatial arrangement ofthe primary structures in each 2D image is determined. An example, howsuch determination may be performed, is given below with reference toFIGS. 6A and 6B. After the spatial arrangement of the primary structuresis determined, a visual representation of the 2D images is generated insubsequent method step S3. The latter visual representation of all or apart of the 2D images is arranged in a 3D visual representation as 2Dslices or 2D planes.

FIG. 5 shows an exemplary embodiment of a system 9 according to anembodiment of the invention. Substantially, necessary for performing thesteps of the method, a processing unit 100 is part of the device.

An exemplary imaging device or imaging unit 200 includes an X-raysource, and an X-ray detector 260, wherein these two units are mountedon a C-arm 220.

Furthermore, the system 9 in FIG. 5 includes an input unit 300, by meansof which for example an intended imaging direction may be manuallyentered. Further, a user can input structures, which shall be consideredas primary structures in the images. Also shown is a connection to adatabase 600, located for example in a network. The database 600 maycomprise information regarding anatomical structures, for example from3D scans of different anatomical structures, so that the imagedanatomical structure may be compared with this information so as todetermine or identify specific anatomical structures. The database mayfurther comprise information regarding a sequence of necessary and/orpossible steps of a surgical procedure. Further, the database cancomprise a storage comprising at least one or a series of earlieracquired reference images. It is noted that it is also possible toautomatically determine the progress of the surgical procedure based ondetectable aspects in an x-ray image, wherein such aspects may be ininstrument and/or implant.

Finally, there is an indication in FIG. 5 of an anatomical structure ofinterest 500 as well as of a reference object 64 formed by a pluralityof radiopaque spheres. Within said anatomical structure, for example abone of a patient may be located which may be subject to the describedprocedures.

With reference to FIGS. 6A and 6B, a method to determine a spatialarrangement of objects in a 2D image is explained in the following.

FIG. 6A shows a reference body formed, in the example, by four spheres640, 641, 642, 643 being arranged in space in a predetermined way.Further shown are lines representing x-ray beams emitted by an x-raysource 240, 241, respectively. Each line ends on one of the projectionsurfaces denoted as AP (anterior-posterior) or ML (medio-lateral). Onthe projection surface ML, the spheres of the reference body form afirst pattern of projection points 640′, 641′, 642′ and 643′, and on theprojection surface AP, the spheres form a second pattern of projectionpoints 640″, 641″, 642″ and 643″. As can be easily seen, the firstpattern on the surface ML differs from the second pattern on the surfaceAP. A skilled person will appreciate that it is possible to arrangespheres of a reference body in three-dimensional space such that aunique projection pattern will be achieved for each projectiondirection. Consequently, it is possible to determine the imagingdirection, based on the detected projection pattern, and to determinethe actual orientation of the reference body in space in relation to theimaging device. Furthermore, as the beams follow a fan angle, thespatial position, i.e. the distances of the references body to the x-raysource and the x-ray detector, respectively, can be calculated based onmeasured distances of the projection points. In fact, it is merely amatter of geometry to calculate the actual position and orientation ofthe reference body based on a single projection of the same.

With the reference body as a “spatial anchor”, it is also possible todetermine an actual position and orientation of an anatomical structurebased on a single x-ray image, as schematically illustrated in FIG. 6B.Here, a projection of a head 500 of a femur, i.e. of a ball head isshown on each of the projection surfaces, wherein the relation of theprojection 500′ to the projections of the reference body on the surfaceML differs from the relation of the projection 500″ to the projectionsof the reference body on the surface AP. This illustrates that theprojections of the reference body and the relation to the anatomicalstructures in the projection image are unique for each imagingdirection. Consequently, the spatial position and orientation of thereference body can be determined and also the spatial position andorientation of the anatomical structure in the vicinity of the referencebody, based on one x-ray image.

While embodiments have been illustrated and described in detail in thedrawings and afore-going description, such illustrations anddescriptions are to be considered illustrative or exemplary and notrestrictive, the invention is not limited to the disclosed embodiments.

Other variations to the disclosed embodiments can be understood andeffected by those skilled in the art in practising the claimedinvention, from a study of the drawings, the disclosure and the appendedclaims. In the claims, the word “comprising” does not exclude otherelements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor or other unit may fulfil thefunctions of several items recited in the claims.

The mere fact that certain measures are recited and mutually differentdependent claims does not indicate that a combination of these measurescannot be used to advantage. The computer program may bestored/distributed on a suitable medium such as an optical storagemedium or a solid-state medium supplied together with or as a part ofanother hardware, but may also be distributed in other forms, such asvia the Internet or other wired or wireless telecommunication systems.Any reference signs in the claims should not be construed as limitingthe scope.

Although the invention herein has been described with reference toparticular embodiments, it is to be understood that these embodimentsare merely illustrative of the principles and applications of thepresent invention. It is therefore to be understood that numerousmodifications may be made to the illustrative embodiments and that otherarrangements may be devised without departing from the spirit and scopeof the present invention as defined by the appended claims.

The invention claimed is:
 1. A method for visualizing a bone, inparticular a femur or a hip bone, the method comprising: providing 2Dimages of the bone wherein at least two of these 2D images are taken atdifferent respective imaging orientations with respect to the bone,detecting outlines of a primary structure in the 2D images of the bone,generating visual representations of the 2D images based on theoutlines, and displaying the visual representations of each of the 2Dimages simultaneously as a plurality of corresponding 2D slices arrangedin a 3D visual representation for supporting a spatial sense of a user,wherein each of the 2D slices is arranged in the 3D visualrepresentation, such that, from a perspective of a viewpoint of the 3Dvisual representation, an angle defined between a surface normal of each2D slice and a surface normal of every other one of the 2D slices is thesame as a respective angle defined between the imaging orientations ofthe corresponding 2D images.
 2. The method according to claim 1, whereinthe primary structure is at least one of a bone and a bone fragment. 3.The method according to claim 1, wherein the primary structure is atleast one of an implant and a reference body.
 4. The method according toclaim 1, further comprising the steps of: detecting at least threemarkers in the 2D images, wherein a marker is one of a reference body, apart of a reference body, an implant or a bone shape, and determining aspatial arrangement of the primary structure based on the position ofthe at least three markers, wherein the step of arranging the visualrepresentation of the 2D images in the 3D visual representation is basedon the determined spatial arrangement.
 5. The method according to claim1, further comprising the step of: classifying the primary structureinto a class of implants, a class of bones, a class of bone fragmentsand/or a class of reference bodies, wherein the step of arranging thevisual representation of the 2D images in the 3D visual representationis based on the classification of the primary structure.
 6. The methodaccording to claim 5, wherein the visualization of the primary structureis limited to at least one of the classes of implants, bones, bonefragments and/or reference bodies.
 7. The method according to claim 1,wherein the 3D visual representation is rotatable showing the visualrepresentations of the 2D images from different viewpoints.
 8. Themethod according to claim 1, wherein the visualization of the primarystructure is based on a detection of a predetermined surgery step,wherein detection of the predetermined surgery step is based on a numberand a position of primary structures in the 2D images.
 9. The methodaccording to claim 1, wherein the 2D images are X-ray images.
 10. Asystem for visualizing a bone, in particular a femur or a hip bone,comprising: a detection unit, and a processing unit, wherein thedetection unit is configured to provide 2D images of the bone, whereinat least two 2D images are taken at different respective imagingorientations with respect to the bone, wherein the detection unit isfurther configured to detect outlines of a primary structure in the 2Dimages of the bone, wherein the processing unit is configured togenerate visual representations of the 2D images based on the outlines,and wherein the processing unit is further configured to display thevisual representations of each of the 2D images simultaneously as aplurality of corresponding 2D slices arranged in a 3D visualrepresentation to support a spatial sense of a user, each of the 2Dslices being arranged in the 3D visual representation, such that, from aperspective of a viewpoint of the 3D visual representation, an angledefined between a surface normal of each 2D slice and a surface normalof every other one of the 2D slices is the same as a respective angledefined between the imaging orientations of the corresponding 2D images.11. A non-transitory computer readable medium encoded with a computerprogram, which, when executed by a processor, performs the method stepsaccording to claim
 1. 12. The method according to claim 1 wherein onlytwo 2D images of a femur are used with the two 2D images taken at anglesof at least 15° from one another.
 13. The method according to claim 12wherein the two 2D images are x-ray images.